Mode Connections For Clinical Incremental Learning: Lessons From The COVID-19 Pandemic (preprint)

Dynamic distribution shifts caused by evolving diseases and demographic changes require domain-incremental adaptation of clinical deep learning models. However, this process of adaptation is often accompanied by catastrophic forgetting, and even the most sophisticated methods are not good enough for clinical applications. This paper studies incremental learning from the perspective of mode connections, that is, the low-loss paths connecting the minimisers of neural architectures (modes or trained weights) in the parameter space. The paper argues for learning the low-loss paths originating from an existing mode and exploring the learned paths to find an acceptable mode for the new domain. The learned paths, and hence the new domain mode, are a function of the existing mode. As a result, unlike traditional incremental learning, the proposed approach is able to exploit information from a deployed model without changing its weights. Pre-COVID and COVID-19 data collected in Oxford University hospitals are used as a case study to demonstrate the need for domain-incremental learning and the advantages of the proposed approach.

A Large Language Modelling Deep Learning Framework for the Next Pandemic (preprint)

Deep neural networks have been integrated into the whole clinical decision procedure which can improve the efficiency of diagnosis and alleviate the heavy workload of physicians. Typical applications include 1) medical report generation, 2) disease classification, and 3) survival prediction. Since most neural networks are supervised, their quality heavily depends on the volume and quality of available labels. However, for novel diseases, e.g., new pandemics or new variants, there are few existing labels. In addition, the acquisition of new pandemic cases to collect sufficient labels for training is time-consuming and is typically unavailable at the early stage. To prepare neural networks for the next pandemic, in this paper, we propose a large language model - Unsupervised Learning from Unlabelled Medical Images and Text (ULUMIT) framework, which can learn broad medical knowledge (e.g., image understanding, text semantics, and clinical phenotypes) from unlabelled data. As a result, when encountering new pandemics, our framework can be rapidly deployed and easily adapted to them with extremely limited labels. Furthermore, ULUMIT supports medical data across visual modality (e.g., chest X-ray and CT) and textual modality (e.g., medical report and free-text clinical note); therefore, it can be used for any clinical task that involves both visual and textual medical data. We demonstrate the effectiveness of our ULUMIT by showing how it would perform using the COVID-19 pandemic ``in replay''. In particular, in the retrospective setting, we test the model on the early COVID-19 datasets; and in the prospective setting, we test the model on the new variant COVID-19-Omicron. The experiments are conducted on 1) three kinds of input medical data, image-only, text-only, and image-text; 2) three kinds of downstream tasks, medical reporting, diagnosis, and prognosis; 3) five public COVID-19 datasets; and 4) three different languages, i.e., English, Chinese, and Spanish. All experiments consistently show that our framework can make accurate and robust COVID-19 decision-support tasks with little labelled data (such as considering information from only one patient), providing an impact on medical data analysis during the early stage of the next pandemic. Besides COVID-19, our framework can be applied to identify 14 common thorax diseases and tuberculosis across five additional public datasets, demonstrating its robustness in generalization and transferability. In brief, our framework achieves state-of-the-art performances on ten datasets.

Evolution Characteristics and Influencing Factors of Global Dairy Trade

There are imbalances and uncertainties in the global supply and demand of dairy products, owing to the adverse influence of overall economic changes, dairy prices, agricultural politics, the COVID-19 pandemic, and severe climate. This paper aims to explore the evolving characteristics and influencing factors of the global dairy trade pattern and make recommendations for the sustainable development of the global dairy trade. This paper studies the evolutionary characteristics of the global dairy trade pattern from the perspective of the overall structure, individual characteristics, and core-periphery structure through complex network analysis (CNA), using the countries involved in dairy trade from 2000 to 2020. Furthermore, this study explores the influencing factors of the dairy trade network using a quadratic allocation procedure (QAP). The results indicate that the global dairy trade network has been expanding, with prominent scale-free features and small-world characteristics. Individual countries display obvious heterogeneity, whereas the core import regions of the dairy shift from Europe, East Asia, and America to North America, the Middle East, and East Asia. Contrary to this, there is no significant change in the core export regions. Consequently, the entire dairy trade network represents a clear core-periphery structure. Moreover, the income per capita gaps, geographic distance gaps, and common language always affect the trade value and dairy trade relations across the countries. Meanwhile, economic level gaps and regional trade agreements have become increasingly significant. Thus, the dairy trade may not follow the 'border effect';. Lastly, this paper also extends recommendations for the sustainable development of the dairy trade.